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£>y We consider the problem of distributed joint source-channel coding of correlated Gaussian sources over a 

Gaussian Multiple Access Channel (GMAC). There may be side information at the decoder and/or at the encoders. 

t^J- ■ First we specialize a general result in [20] to obtain sufficient conditions for reliable transmission over a Gaussian 

MAC. This system does not satisfy the source-channel separation. We study and compare three joint source-channel 

f-H ■ coding schemes available in literature. We show that each of these schemes is optimal under different scenarios. 

One of the schemes, Amplify and Forward (AF) which simplifies the design of encoders and the decoder, is optimal 

O : 

at low SNR but not at high SNR. Another scheme is asymptotically optimal at high SNR. The third coding scheme 
is optimal for orthogonal Gaussian channels. We also show that AF is close to the optimal scheme for orthogonal 
channels even at high SNR. 
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j>! ; I. Introduction and Survey 

Sensor networks are used in a wide variety of applications, the most common being the spatio-temporal 

a | 

monitoring of a random field ([1]) and the detection of change in its statistics ([23]). Sensor nodes are 
inexpensive with limited battery power and storage and hence have limited computing and communication 
capabilities ([1]). These nodes transmit their observations to a fusion center to estimate the sensed random 
field. Since transmission is very energy intensive, it is important to minimize it. 

The sensor nodes transmit their observations to the fusion center (or a cluster head) usually over 
a Multiple Access Channel (MAC) ([2], [23]). Often the received symbol is a super-position of the 
transmitted symbols corrupted by Additive White Gaussian Noise (AWGN). This then is the well known 
Gaussian MAC (GMAC). This channel is interesting from a practical as well as a theoretical perspective. 

Preliminary versions of parts of this paper appear in Allerton 07, ISITA 08 and WCNC 09. R Rajesh and Vinod Sharma are with the 
Dept of Electrical Communication Engineering, IISc, Bangalore, India. Email: rajesh@pal.ece.iisc.ernet.in, vinod@ece.iisc.ernet.in. 



Also the sensor nodes can be modeled as discrete or continuous sources. For continuous sources, Gaussian 
distribution is particularly useful. This for example can happen if the sensor nodes are sampling a Gaussian 
random field. We can come across it in the problem of detection of change also. Then, it is often the 
detection of change in the mean of the sensor observations with the sensor observation noise being 
Gaussian ([23]). Thus, in this paper we focus on transmission of Gaussian sources over a GMAC. 

One common way to use the MAC is via Time division multiple access (TDMA), Frequency division 
multiple access (FDMA) or Code division multiple access (CDMA) ([3], [7]). These protocols although 
suboptimal are used due to practical considerations. These protocols make the MAC a set of parallel 
orthogonal channels (for CDMA, it happens if we use orthogonal codes). Thus we will also consider 
orthogonal Gaussian channels. 

In the following we survey the related literature. Cover, El Gamal and Salehi [6] provided sufficient 
conditions for transmitting losslessly discrete correlated observations over a discrete MAC and show that 
source-channel separation does not hold for this system. The results of [6] have been extended in [20] to 
the case of lossy transmission with side information and continuous alphabets. 

The distributed Gaussian source coding problem is discussed in [15], [24]. The exact rate region for two 
users is provided in [24]. In [14] one necessary and two sufficient conditions for transmitting a bivariate 
jointly Gaussian source over a GMAC are provided. The authors prove that the (uncoded) amplify and 
forward (AF) scheme is optimal below a certain SNR. 

In [17] it is shown that feedback increases the capacity of a GMAC. GMAC with lossy transmission 
of correlated discrete sources and side information is studied in [19]. Transmission of correlated jointly 
Gaussian input over a GMAC is also studied in [18] and the edge capacities obtained are similar to the 
expressions in [17]. In [10] the authors discuss a joint source channel coding scheme over a MAC and 
show the scaling behavior for the Gaussian channel. Actually their problem is closer to the Gaussian CEO 
problem [16]. The scaling laws for the problem without side information are discussed in [12] and it is 
shown that separating source coding from channel coding may require exponential increase in bandwidth, 
as the number of sensors increases. In [13], the authors show that for a Gaussian sensor network it is 
better to compress the local estimates than to compress the raw data. 

Conditions for separation to hold in multiple access channels are given in [21]. Separation holds for 
a GMAC under receiver power constraints ([9]). In general, for orthogonal channels separation holds for 
lossless ([3]) as well as for lossy transmission ([26]). It is shown in [11] that separation holds and uncoded 



transmission achieves capacity in a Gaussian relay network as the number of relays go to infinity. 

This paper makes the following contributions. From our general results in [20] we obtain explicit 
conditions for transmission of correlated Gaussian sources with given distortion over a GMAC. Also we 
compare the two schemes in [14] with a separation based scheme. We explicitly show that the AF scheme 
in [14] is not optimal at high SNR. However another scheme studied in [14] is. Furthermore, for AF, it 
may not be optimal to use all the power. We provide the results with side information also. We identify 
an optimal coding scheme for transmission of correlated Gaussian sources over orthogonal Gaussian 
channels and compare it to AF. We show that AF is close to the optimal scheme. We also compare 
the performance with side information. The results are extended to more than two users providing some 
unexpected conclusions. 

The paper is organized as follows. Sufficient conditions for transmission of continuous correlated sources 
over a continuous MAC are given in Section II. The transmission of Gaussian sources on a Gaussian MAC 
is discussed in Section III. Different joint source-channel coding schemes for transmission are studied 
and their asymptotic performances are compared. In Section IV optimal power allocation to minimize 
the sum of the distortions for the three schemes is obtained. Section V gives the performance with side 
information. In Section VI transmission of correlated Gaussian sources over orthogonal Gaussian channels 
is considered. The performance of the AF scheme is studied VII. Section VIII concludes the paper. 

II. Transmission of correlated sources over a MAC 

In this section we consider the transmission of memoryless dependent sources, through a memoryless 
multiple access channel. The sources and/or the channel input/output alphabets can be discrete or 
continuous. Furthermore, side information about the transmitted information may be available at the 
encoders and the decoder. Thus our system is very general and covers many systems studied earlier. 

We consider two sources (U 1: U 2 ) and side information random variables Z 1: Z 2 , Z with a known joint 
distribution F(u 1 ,u 2 , Zi, z 2 , z) (generalization to multiple sources is also available). Side information 
Zi is available to encoder i, i = 1,2 and the decoder has side information Z. The random vector 
sequence {(U ln , U 2n , Z ln , Z 2n , Z n ),n > 1} formed from the source outputs and the side information with 
distribution F is independent identically distributed (iid) in time. We will denote {Uik, k — 1, n} by 
[/". Similarly for other sequences. The sources transmit their codewords X in 's to a single decoder through 
a memoryless multiple access channel. The channel output Y has distribution p(y\xi,x 2 ) if x x and x 2 
are transmitted at that time. Thus, {Y n } and {(X in , X 2n )} satisfy p(yk\y k ~ 1 , x\, x 2 ) = p(yk\x±k, x 2 k). The 



decoder receives Y n and also has access to the side information Z n . The encoders at the two users do 
not communicate with each other except via the side information. The decoder uses the channel outputs 
and its side information to estimate the sensor observations U in as U in , i — 1,2. It is of interest to 
find encoders and a decoder such that {U ln: U 2n ,n > 1} can be transmitted over the given MAC with 
E[di(Ui, Ui)\ < D\ and E[d 2 (U 2 , U 2 )\ < D 2 where di are non-negative distortion measures and _Dj are 
the given distortion constraints. If the distortion measures are unbounded we assume that u*, i G {1,2} 
exist such that E[di(U i: u*)] < 00, % — 1,2. Source channel separation does not hold in this case. 

Definition: The source (£/", U 2 ) can be transmitted over the multiple access channel with distortions 
D=(Di, D 2 ) if for any e > there is an n such that for all n > n there exist encoders ■ : x Z™ — > 



< Di + e, i = 1,2 



X?, % = 1, 2 and a decoder f%-.y n xZ n ^ (U1M2) such that \E £? =1 d(U ij: U^) 
where (C>r,C/ 2 n ) = f D {Y n ,Z n ) and Z^, Z, X u y, U { are the sets in which U h Z u Z, X h Y, U, t 
take values. 

We denote the joint distribution of (Ui,U 2 ) by p(ui,u 2 ). X <-> Y <-> Z will indicate that {X, Y, Z} 
form a Markov chain. 

The proof of the following theorem is available in [20] for more than two users also. It will be specialized 
to the Gaussian sources and GMAC and then used to obtain efficient joint source-channel coding schemes. 



Theorem 1: A source (Ui,U 2 ) can be transmitted over the multiple access channel with distortions 
(Di,D 2 ) if there exist random variables (Wi,W 2 , X 1 , X 2 ) such that 

(1) p(ux, u 2 , z 1 , z 2 , z, wi, w 2 , x 1 ,x 2 , y) = p(u u u 2 , z u z 2 , z)p(w 1 \u 1 , z^w^, z 2 ). 

p(x 1 \w 1 )p(x 2 \w 2 )p(y\x 1 , x 2 ). 

(2) There exists a function fjj : W± x W 2 x Z — > (Mi x U 2 ) such that E[d{U u fy)] < D h i = 1, 2, where 
(Ui, U 2 ) = /d(Wi, Z), Wj are the sets in which Wi take values and the constraints 

I(U 1 ,Z 1 ;W 1 \W 2 ,Z) < I(X 1 ;Y\X 2 ,W 2 ,Z), 

I(XJ 2 ,Z 2] W 2 \W U Z) < I(X 2 ;Y\X 1 ,W 1 ,Z), (1) 
I{U U U2,Z U Z 2 -W U W 2 \Z) < I[X X ,X 2 -Y\Z), 



are satisfied. 



If the channel alphabets are continuous (e.g., GMAC) then in addition to the conditions in Theorem Q] 
certain power constraints -E 1 ^ 2 ] < Pi, i = 1, 2 are also needed. In general, we could impose a constraint 
E[gi(Xi)] < (Xi where g± is some non-negative cost function. Furthermore, for continuous alphabet r.v.s ( 
sources/channel input/output) we will assume that probability density exists so that one can use differential 
entropy. 

The correlations in (Ui, U 2 ) and the side information [Z\, Z 2 , Z) are used to decrease the left side and 
increase the right side in (OQ). 

We specialize this result to the Gaussian sources and GMAC in the next section. The main problem 
in applying the result in the above theorem in specific examples is to obtain a good coding scheme 
(Wi, W 2 , X\,X 2 ) (as is usual in most information theoretic results). Thus we will also consider specific 
coding schemes and study their performance. 

In the first part of this paper we will mostly consider the system without side information Z\, Z 2 , Z. 
Then the inequalities in (OQ) become 

I(Ux; W X \W 2 ) < 7(X i; Y\X 2 , W 2 ), I(U 2 ; W 2 |Wi) < I(X 2 ; Y\X h W x ), 

I(U X , U 2 ; W 1} W 2 ) < I(X h X 2 ; Y). (2) 

It has been shown in [19] that under our conditions I{X X ; Y\X 2 , W 2 ) < I(Xi,Y\X 2 ) and I(X 2 ; Y\X U W x ) 
< I(X 2 ;Y\Xi). We will use these relaxed upper bounds to obtain good joint source-channel coding 
schemes. 

III. Gaussian sources over Gaussian MAC 

In a Gaussian MAC the channel output Y n at time n is given by Y n = X\ n + X 2n + N n where X\ n and 
X 2n are the channel inputs at time n and N n is a Gaussian random variable independent of X ln and X 2n , 
with E[N n ] = and var(N n ) = a 2 N (we will denote this distribution by A/"(0, &%))■ We will also assume 
that (Ui n , U 2n ) is jointly Gaussian with mean zero, variances of, i — 1,2 and correlation p. The distortion 
measure will be Mean Square Error (MSE). The transmission power constraints are -EfX 2 ] < Pi, i = 1, 2. 

It is shown in [20] that I(Xi,Y\X 2 ), I(X 2 ;Y\Xx) and I(Xi, X 2 ;Y) are maximized by zero mean 
Gaussian random variables (Xi,X 2 ) with variances Pi and P 2 . Also if correlation between X% and X 2 



is p then the conditions in © (after the relaxation mentioned below ©) become 



I(Ui;Wi\W 2 ) < 0.5 log 



1 + 



A(W 
a N 2 



X2\ 



I(U 1 ,U 2 ;W 1 ,W 2 ) < 0.5 log 



I(U 2 ;W 2 \W 1 ) < 0.5 log 
Pi + P 2 + Tp^fT\P 2 



1 + 



m - p 

a N 2 



X2\ 



1 + 



a N 2 



(3) 



An advantage of the relaxed conditions © is that we will be able to obtain a good source-channel 
coding scheme. Once (X 1? X 2 ) are obtained we can check for sufficient conditions ©. If these conditions 
are not satisfied then we increase p till © is satisfied. See more details in [19]. 

In the rest of the paper we consider three specific coding schemes to obtain Wi, W 2 , X 1; X 2 where 

(WiiW^) satisfy the distortion constraints in Theorem \T\ and (Xi,X 2 ) are jointly Gaussian with an 

appropriate p such that © (and then ©) is satisfied. These coding schemes have been available before. 

Our purpose is to compare their performance. More importantly we show that each of these coding schemes 

can be optimal under different scenarios. Also in two of these coding schemes Xi is obtained from Wi 

by scaling. Thus, the relaxed conditions © in fact are identical to the sufficient conditions obtained 

in Theorem 1. In the third coding scheme (SB), W\ and W 2 are (asymptotically) independent due to 

Slepian-Wolf coding. Thus again I{X x] Y\X 2 , W 2 ) = J(X i; Y\X 2 ) and I(X 2 ; Y\X U W x ) = I(X 2 ; Y\X{). 

A. Amplify and forward scheme 

In the Amplify and Forward (AF) scheme the channel codes Xi are just scaled source symbols Ui, Since 

(Ui, U 2 ) are themselves jointly Gaussian, (X 1; X 2 ) will be jointly Gaussian and retain the dependence of 

inputs (Ui, U 2 ). The scaling is done to ensure P^j 2 ] = Pi,i = 1, 2. For a single user case this coding is 

optimal ([7]). 

At the decoder inputs Ui and U 2 are directly estimated from Y as Ui = E[Ui\Y], i = 1,2. Because 
Ui and Y are jointly Gaussian this estimate is linear and also satisfies the Minimum Mean Square Error 
(MMSE) and the Maximum Likelihood (ML) criteria. 

The MMSE distortion for this encoding-decoding scheme can be easily shown to be 

w= ^i 2 IP 2 (1-p 2 ) + ^ 2 ] ^_ a 2 2 [Pi(l-p 2 ) + ^ 2 ] 
1 P l + P 2 + 2p^fl\P2 + <yN 2 ' 2 P 1 + P 2 + 2p^/P[P 2 + a N 2 ' 

Since encoding and decoding require minimum processing and delay in this scheme, if it satisfies the 
required distortion bounds Di, it should be the scheme to implement. This scheme has been studied in [14] 
and found to be optimal below a certain SNR for two-user symmetric case (Pi = P 2 , o\ = a 2 , Di = D 2 ). 
However unlike for single user case, in this case user 1 acts as interference for user 2 (and vice versa). 



Thus one should not expect this scheme to be optimal under high SNR case. Also, in the asymmetric 
case, it is not clear if all the power P 1 , P 2 needs to be expended for AF transmission. We will address 
these issues in the following sections. 

B. Separation based scheme 

In separation based (SB) approach the jointly Gaussian sources are vector quantized to and W 2 - The 
quantized outputs are Slepian-Wolf encoded [22]. This produces code words, which are (asymptotically) 
independent. The rate (Ri,R 2 ), distortion (Di,D 2 ) pairs achievable for this source coding are ([24]) 



The independent code words (Wf, W 2 ) are encoded to capacity achieving independent Gaussian channel 
codes (X™, X 2 ). This is a very natural scheme and has been considered by various authors ([6], [7], [21]). 

Since source-channel separation does not hold for this system, this scheme is not expected to be optimal 
(however we will see that it is optimal for orthogonal channels). But because this scheme decouples source 
coding from channel coding, it is preferable to a joint source-channel coding scheme with comparable 
performance. 

C. Lapidoth-Tinguely scheme 

In this scheme, obtained in [14], (C/J 1 , U 2 ) are vector quantized to 2 nRl , 2 nR2 (f/J 1 , U 2 ) vectors where 
Ri and R 2 will be specified below. Also, Wi,W 2 are 2 nRl and 2 nR2 , n length code words obtained 
independently with distributions A/"(0, 1). For each w™, we pick the codeword w? that is closest to it. This 
way we obtain Gaussian codewords W™, W 2 which retain the correlations of ({7™, U 2 ). X™ and X 2 are 
obtained by scaling W™, W 2 to satisfy the transmit power constraints. Thus © is a sufficient condition 
for this scheme. We will call this scheme LT . (Ui, U 2 , Wi, W 2 ) are (approximately) jointly Gaussian with 
covariance matrix 



R, > 0.51og[<T 2 1 (l-p 2 + p 2 2- 2R 2)/D 1 ) 
R 2 > 0.51og[a 2 2 (l-p 2 + p 2 2- 2Rl )//J 2 ] 



(5) 



Rt + R, > 0.5\og[a 2 a 2 2 (l-p 2 )P(D 1 ,D 2 )/2D 1 D 2 ], 



where p{D u D 2 ) = 1 + y/l + 4p 2 /J 1 /J 2 /a 2 ( T 2 (l - p 2 ) 2 . 





a 2 {l-2~ 2R i) 

V<7 1( 7 2 (l-2- 2 «2) 



p CTl( r 2 (l-2- 2 «i) 
a 2 (l-2- 2 ^ } 



(6) 



a 2 (l-2- 2 ^) J 



where p = p^J (1 — 2~ 2Rl )(l — 2~ 2R2 ). The required (Ri,R 2 ) are obtained from © as follows. From 
I(Ui; Wi\W 2 ) = H(W 1 \W 2 ) - H(Wi\W 2 , U-f) and the fact that the Markov chain condition W 1 <-> U x <h> 
U 2 holds, we obtain tf(Wi|W 2 ,£/i) = H{Wi\U x ) and 7([/ i; WilWb) = 0.5 log [(1 - p 2 )2 2Rl ] . 

Thus from © (which is now the same as © because Xj are obtained from PVj via scaling) we need 
i?i and i? 2 which satisfy 



Rx < 0.5 log 



A + 1 

o 1 



o N * [1 - p 2 
Ri + R 2 < 0.5 log 



R 2 < 0.5 log 



+ 



a N 2 + P x + P 2 + 2p^I\P 2 



(7) 



(1 - P 2 W 2 

The inequalities © are the same as in [14]. Thus we recover the conditions in [14] from our general 
result ©. 

Taking Uj, = E[Ui\Wi, W 2 ), i = 1, 2, we obtain the distortions 

<7i 2 2- 2fl i [l-p 2 (l-2- 2i?2 )] 



Dx = var(U 1 \W 1 ,W 2 ) 
D 2 = var(U 2 \W 1 ,W 2 ) 



(1 - p 2 ) 
a 2 2 2- 2R * [1-p 2 (l-2- 2R i)} 



(8) 
(9) 



(1 - p 2 ) 

The minimum distortion is obtained when p is such that the sum rate is met with equality in ©. 

D. Asymptotic performance of the three schemes 

We compare the performance of the three schemes. We will extend the comparison to more than two 
users in Section IIII-EI For simplicity we consider the symmetric case: P\ = P 2 = P, o x = a 2 = a, 
Di = D 2 = D. We will denote the SNR P/a N 2 by S and D(S) will denote the distortion for a given S. 

Consider the AF scheme. From ©, 

a 2 [S(l-p 2 ) + l] 



D(S) 



25(l + p) + l 



(10) 



Thus D(S) decreases to a 2 (l — p)/2 strictly monotonically at rate 0(1) as S — >• 00. Also, 



lim 

5^0 



D(S) 



a 



s 



(11) 



Hence, D(S) -> a 2 at rate O(S) as S -> . 

Next consider the SB scheme. From [24] if each source is encoded at rate R, it can be decoded at the 
decoder with distortion 

D = ^2-^(1 - p 2 ) + p 2 2~ SR . (12) 



At high SNR, from the capacity result for independent inputs, we have R < 0.25 log S ([7]). Then from 
(PT2l) we obtain 

(T 4 (l - p 2 ) ffV 

s + H~ 2 



D>\l ' + (B) 



and this lower bound is achievable. As S — > oo, this lower bound approaches zero at rate 0(\/S). Thus 
this scheme outperforms AF at high SNR. At low SNR, R fa S/2 and hence from (fl~2l 



7J > pV 4 2- 4S + (j 2 (l -p 2 )2" 2S . (14) 

Thus Z) — >■ a 2 at rate 0(S 2 ) as 5 — > at high p and at rate O(S) at small p. Therefore we expect that 
at low SNR, at high p this scheme will be worse than AF but at low p it will be comparable. 

Consider the LT scheme. In the high SNR region we assume that p = p since R = R 1 = R 2 are 
sufficiently large. Then from © R 0.25log[2S/ (1 — p)\ and the distortion can be approximated by (and 
asymptotically approaches) 

D w (7 2 V(1 - p)/25. (15) 

Therefore, D — >• as S 1 — > oo at rate 0{yS). This rate of convergence is same as for SB. However, the 
right side in (fT3l) is greater than that of (fT5l) and at low p the two are close. Thus at high SNR LT always 
outperforms SB but the improvement is small for low p. Now we show that LT is in fact asymptotically 
optimal at high SNR. 

The necessary conditions (NC) to be able to transmit on the GMAC with distortion (D, D) for the 
symmetric case are [14], [25] 

" ^ (16) 

25(l+p)+l ' iU1 ° ^ 1-fP- 

At high SNR the lower bound tends to ct 2 a/(1 — p)/2S. This is same as the expression for LT in (fT5l) . 
At low SNR 




4 



and evaluating D from ([8]) we get 



a 2 ' 



2 2- s (l-p 2 (l-^/r^ 2 2~ 1 
D = ^ '- (17) 



where S = S(l + p). Therefore D -> a 2 as S -> at rate 0(S' 2 ) at high p and at rate 0(5) at low p. 
These rates are the same as that for SB. In fact, dividing the expression for D at low SNR for SB by that 
for LT, we can show that the two distortions tend to a 2 at the same rate for all p . 

The above three schemes along with the necessary conditions are compared below using exact 
computations. Figures Q] and [2] show the distortion as a function of SNR for unit variance jointly Gaussian 
sources with correlations p = 0.1 and 0.75. 

From these plots we confirm our theoretical conclusions provided above. In particular we obtain that: 

• AF is close to necessary conditions and hence optimal (as shown in [14]) at low SNR. The other 
two schemes perform worse at low SNR. 

. SB and LT perform better than AF at high SNRs. 

• LT performs better than SB in general. 

• Performance of SB and LT are close for low p and for high p at low SNR. 

• LT is close to optimal at high SNR and is asymptotically optimal. 

E. Multiple users 

In this section we study the performance of the three schemes for more than two users. This is 
important for sensor networks. We limit ourselves to symmetric scenario (this is for convenience, the 
general case can be handled similarly). Consider jointly Gaussian (Ui, U2, Z7jy) having mean zero 
vector with E(U i ,Uj)=p ) i 7^ j and E[U 2 } = 1. Let all the nodes be (average) power constrained to P. 

Consider the AF scheme. Let Daf(N, P) denote the distortion per user incurred in transmitting the 
sources through a GMAC. For this scheme then 

D AF (N, P) - 1 - NP{1 + {N _ 1)p) + a 2- 



Also, 



N - 1 

lim D AF (N, P) = l-p, lim D AF (N, P) = — — (1 - p). (19) 

N— >oo P^oo iV 



From (1191 ) we can see that distortion per user tends to 1 — p for all P. Thus correlation reduces the 
asymptotic mean distortion. Also from (fT9l) we see that, at high SNR, distortion per user increases from 
(1 — p)/2 for the two user case to 1 — p as N — > 00. 

We plot D A f(N, P) for p = 0.8 in Fig. |3j Interestingly, we find that for low SNR as N increases 
Daf(N,P) decreases. But at high SNR as N increases Daf(N,P) increases. This happens because 



of the trade-off between the interference caused by many users and the beamforming gain (because of 
the correlation in the observations). This trade-off depends on the SNR. The cut-off where this change 
(decrease to increase) occurs depends on p. The cut-off increases with p. For p = 0.8 and for N < 10 the 
cutoff could be taken as 3dB . Thus, for iV < 10, for P < 3dB distortion D(N, P) decreases with N. 

Consider the SB scheme. In this coding scheme the source output U^={Un, ...,Ui n ), i = 1,...N is 
vector quantized to by sensor node Si. The W^'s are Slepian-Wolf encoded. The encoded data is 
sent over the channel using independent channel codewords. W"'s are losslessly obtained at the decoder 
and then U^'s are estimated. In a similar way LT can be extended to N users. 

We compare the performance of AF, SB and LT for N = 2, 3 in Fig @J We see that AF performs well 
over a larger SNR region compared to the other two schemes as N increases. 



IV. Optimal power allocation 

Till now we studied the performance of the three coding schemes for given average powers Pi and P 2 . 
The question arises if indeed it is optimal to expend the whole of P 1 and P 2 or one could do better by 
using lower average powers. Of course if the two users are independent of each other then we should 
expend full powers P x and P 2 . In this section, using the performance obtained in previous sections we 
address this issue for the correlated case. 

Without loss of generality we take the sources with unit variance (when they have different variances, 
by scaling we can reduce the problem to equal, unit variance case). Also, let MAC noise power a 2 N = 1. 
The objective is to find for the three schemes, the minimum distortion for the given power constraints Pi 
and P 2 and also the optimal powers. 

A. Amplify and forward scheme 

For the AF, we find powers a* and b* that minimize 

ftJl+A i* = ft ^fV ft + 1 - (20) 

a + b + 2y/abp + 1 a + b + 2y/abp + 1 

subject to a < Pi and b < P 2 , where /3i,/3 2 are given positive weights. 

Unlike for a single user case, the optimal powers a* and b* may not be equal to Pi and P 2 respectively. 
Therefore, we solve this optimization problem using the Kuhn-Tucker first order conditions ([4]). In the 
following we also study the qualitative behavior of the cost function to get more insight into the problem. 



From Section IIII-DI for symmetric case we know that P strictly decreases as a function of P. Hence 
then for (3x — (3 2 , a* = b* = P. In general, if we use powers a and b, then 

Dl + B2 >^±M1_£)>1^!. 1 _ P . 

a + b + 1\Jabp 1 + P 



The second inequality follows because a + b > 2\/ab and equality is achieved only if a = b. The first 
inequality will tend to equality if a or b — > oo . Thus for large P\ and P 2 , a* ~ 6* ~ min(Pi, P2). The 
minimum Pi + P 2 approaches the lower bound 1 — p as P\ and P 2 — > 00 . However if we keep a fixed 
and increase b then the first lower bound starts increasing after sometimes. Thus if Pi is fixed and P 2 is 
increased, b* will not keep increasing. We will elaborate on it below via explicit computations. 

From (|20l) we find that D\ is monotonically decreasing with a. Next fix a and vary b. If a and b are 
large compared to 1 and if a <C b then Pi « 1 — p 2 . Thus Pi is insensitive to b if 1 <C a <C 6. This can 
be interpreted as follows. At high b the noise can be neglected and when 1 <C a <C b at the decoder P 2 
is available accurately and K will be close to P 2 . Thus P[Pi|P 2 ] will be close P[Pi|F] and the mean 
square error E (U\ — Pi) 2 ~ 1 — p 2 . By symmetry, same follows for P 2 when 1 <^ b <^ a. 

Next consider Pi when 1 <C & < a. Then Pi w — Here Pi is sensitive to b and increases 

a 

with b irrespective of p. This may be because the interference effect of P 2 dominates as b increases. 

Now we again consider the case where Pi is fixed and P 2 is varying. The optimal values 
of a*,b* and P are provided in Table U for p = 0.5 via Kuhn-Tucker conditions. As we see, 
b* is less than P 2 for large P 2 . This happens because when a is fixed, Pi + P 2 , as a func- 
tion of b, is strictly decreasing in the interval [0, c) and strictly increasing in (c, 00) where c = 
[((1 + p 2 ) + + p 2 ) 2 + Aap 2 {l - p 2 )[a{\ - p 2 ) + 2])/2V5p(l " P 2 )?- 

Thus b = c is the unique global minimum of Pi + P 2 for a given a. From Table U we observe that for 
p = .5 and a = 10, c = 17.01. Table U also shows that for P 2 > P 1; a* = P x . 



B. Separation based scheme 

For the SB scheme, Xi,X 2 are independent zero mean Gaussian random variables. The capacity region 
for this case monotonically increases with Pi, P 2 . Thus, the distortion D\ + P 2 will be minimized at the 
powers (a*,b*) = (Pi,P 2 ). 



C. Lapidoth-Tinguely Scheme 



The optimization problem for the LT scheme is: Minimize 



, < 7?2-^[l-p 2 (l-2-^)] , qj2-^[l-p 2 (l-2-^)] 

1^ + /?2 T=p (21) 



Subject to, a < P\, b < P 2 and 

/ 1 \ / 1 \ ( 1 a + b + 2Vabp\ 
< 0.5 log [a + T -^ J J , R 2 < 0.5 log (^6 + , /?.! + iJ 2 < 0.5 log ^_ - + — g j . 

From the constraints we find that Ri, R 2 and Ri + R 2 are monotonically increasing in a, b and p. Hence 
these are maximized at (a, b) = (Pi,P 2 ). Also the objective function is monotonically decreasing in R 1 
and R 2 (the first derivative of the objective function is negative for all non-negative(i?i, R 2 ) for all positive 
Pi, f3 2 ). Hence the minimum distortion is achieved at powers (Pi,P 2 ). 

V. Side information 

In a sensor network the cluster head, to which the neighbouring nodes may transmit their information 
via a MAC, may also be sensing. The sensed observations at the cluster head may be correlated to the 
data being transmitted by other nodes. This leads to the situation where the decoder has side information 
which can help reduce the transmitted data (see Theorem 1). The side information may also be available 
at the encoders because the sensor nodes transmitting their data can also listen to the data transmitted by 
other nodes to the cluster head. Thus in this section we compare the performance of the three schemes 
when there is side information at the encoders and/or the decoder. 

Side information Z { is available at encoder i, i = 1,2 and Z is available at the decoder. One use 
of the side information Zi at the encoders is to increase the correlation between the sources. In our 
setup, then will be a function of [U^Z^). It is known that if random vectors (Vn, Vi p ) and 
(V 2 i, V 2q ) have a joint normal distribution then the functions /(Vn, Vi p ), g(V 2 i, V 2q ) having 
maximum correlations among themselves are linear (see [5]). This motivates us to take appropriate linear 
combinations Lj = c^f/j + biZi, i = 1,2 of (Ui, Z^) at encoder i to generate Wj. When side information 
is available at the decoder it is used to estimate the sources and also it can reduce the rates through the 
channel (as can be seen from (OQ)). Using these ideas in the following we propose techniques to extend 
the three schemes to the case when there is side information at the encoders and the decoder. These 
techniques can be specialized to the case when there is encoder only or decoder only side information. 



A. AF with side information 

Linear combination of the source outputs and side information Li — aJJi + biZi, i = 1,2 is amplified 
to Xi to meet the power constraints and sent over the channel. These linear combinations can increase the 
correlation between the channel alphabets. However a distortion may be incurred in estimating the sources 
from Lj. Thus the optimal a«s and b^s selected may not give the highest possible correlation between 
L\ and L 2 . The decoder side information Z is used to get better estimates of the sources. Hence we 
find the linear combinations, which minimize the sum of distortions. For this we consider the following 
optimization problem: find (01,61,02,62) that 

Minimize D{a u h, a 2 , b 2 ) = E[{JJ X - f>i) 2 ] + E[(U 2 - U 2 ) 2 } (22) 

subject to E[Xf) < P { where ft = E[Ui\Y, Z), i = 1, 2. 

B. SB with side information 

For a given (Li,L 2 ), we use the coding-decoding scheme described in Section UlI-BI The availability 
of the side information Z at the decoder reduces the rates (i?i, R 2 ) at which the vector quantizers operate. 
Side information Z is also useful in estimating the sources. The linear combinations L\ and L 2 are 
obtained which minimize (1221) for this coding-decoding scheme. 

C. LT with side information 

For a given (Li,L 2 ), we use the encoding-decoding scheme described in Section UlI-CI by replacing 
(U™,U 2 ) by (L^Lg). The rates at which the vector quantizers operate are a function of the side 
information at the decoder and are obtained from ©. This side information is also used in estimating 
(Ui,U 2 ) from (Wi, W2). The linear combinations L\ and L 2 are obtained which minimize (T22l) for this 
encoding-decoding scheme. 

D. Comparison of the schemes with side information 

We provide the comparison of the three schemes for U\,U 2 ~ A/"(0, 1) and correlation p. Also for 
illustration purposes, we take the side information with a specific structure which seems natural in this set 
up. Let Z\ = S\U 2 + V\ and Z 2 = s 2 U\ + V 2 , where Vi, V 2 ~ jV(0, 1) and are independent of each other 
and independent of the sources, and s\ and s 2 are constants that can be interpreted as the side-channel 
SNR. This can also be interpreted in the framework of co-operative communication where Z\ and Z 2 
represent the co-operation between the encoders. We also take Z = (Zi, Z 2 ). 



For simplicity, we consider the symmetric case: Pi = P 2 = P, s\ = s 2 , P>\ = D 2 . The channel noise 
variance a 2 N = 1. Thus P can be interpreted as the channel SNR. We obtain the optimal a i; 6j for the 
three schemes and compare the distortions as in Section III. 

Consider the AF scheme. Fig. |5] gives the minimum sum of distortions achieved for p = 0.1 for different 
channel SNRs under various assumptions on the availability of the side information. It can be seen from 
the figure that decoder only side information is much more useful than encoder only side information. The 
reduction in distortion is directly proportional to the quality of the side information (i.e., the side channel 
SNR). The encoder only side information case shows marginal improvement over the no side information 
case. It is also found that when side information is provided at the decoder, providing it at the encoders 
also does not help much as seen in Fig. [5] Thus the critical information is the side information at the 
decoder. This is consistent with the findings in [8]. 

For SB and LT schemes also we find that decoder side information is the most useful one. It is also 
found that for the encoder only side information LT performs better than SB under all SNRs and all p. 
This conclusion is similar to the no side information case. AF performs better than both the schemes for 
low SNR's. These curves are not provided due to lack of space. 

Decoder-only side information is used for estimation at the decoder and also for determining the 
communication rates at the encoders. Comparison of the three schemes for this case is shown in Fig. [6j It 
is seen that SB becomes better than LT when the quality of the side information provided at the decoder 
becomes better. In the symmetric case this cut-off (where SB takes over LT) side channel SNR is a 
function of p. Fig. [6] plots the sum of distortions vs the side channel SNR si = s 2 = s and gives the 
cut-off side channel SNR above which the SB is optimal when p = 0.5. This cut-off is valid for all 
channel SNR's. SB becomes optimal because the estimation efficiency and the communication rates can 
be improved in SB when the quality of the side information improves. The same effect is found when 
side information is fed to both encoders and the decoder (results not shown). 

Overall, for the case of side information, we obtain the following conclusions. Decoder only side 
information is much more useful than encoder only side information. Encoder only side information helps 
marginally if at all. The reduction in distortion is proportional to the side information quality. Also, AF 
is optimal at low SNR with or without side information. However distortions in AF do not go to zero, 
when channel SNR is increased, with or without side information. But distortions for SB and LT do go 
to zero as channel SNR increases. Therefore, at high enough SNR AF is the worst. This is because of 



the interference between the two users. LT is always better than SB in the no side information case. But 
with side information SB is sometimes better than LT. 

VI. Transmission of correlated sources over orthogonal channels 

For the orthogonal channel system Y = (Y x , Y 2 ) and the channel transition matrix is p(yi, ^l^i, = 
p{yi\xi)p{y2\X2)- Then the conditions (OQ) become 

I(U 1 ,Z 1 ;W 1 \W 2 ,Z)<I(X 1 ;Y 1 \W 2 ,Z)<I(X 1 ;Y 1 ), (23) 
I(U 2 , Z 2 ; W 2 \W 1 ,Z)< I(X 2 ; Y 2 \W U Z) < I(X 2 ; Y 2 ), (24) 
I(U U U 2 , Z u Z 2 ; W 1 , W 2 \Z)< I(X U X 2 ; Y x , Y 2 \Z) < I(X x] Y 1 ) + I(X 2 ; Y 2 ). (25) 

Using Fano's inequality, for lossless transmission of discrete sources over discrete channels with 
side information, we can show that the outer bounds in (|23l) -d25T) are in fact necessary and sufficient 
conditions. The outer bounds in (|23l) -d25l) are satisfied with equality if the channel codewords (Xi,X 2 ) 
are independent of each other. Also, the distribution of (Xi, X 2 ) maximizing these bounds is not dependent 
on the distribution of (Ui, U 2 ). Furthermore, the left side of the inequalities are simultaneously minimized 
when W\ and W 2 are independent. Thus, the source coding of (Ui, Z{) and (U 2 , Z 2 ) can be done as in 
Slepian-Wolf coding but also taking into account the fact that the side information Z is available at the 
decoder. 

If we take W x = XJ X and W 2 = U 2 and the side information (Z x , Z 2 , Z)L{U X , U 2 ) (X1Y denotes that 
X is independent of Y) we can recover the conditions in [3]. 

A. Gaussian sources over orthogonal Gaussian channels 

Now we consider the transmission of jointly Gaussian sources over orthogonal Gaussian channels. 
Initially it is assumed that there is no side information Z\, Z 2 , Z. 

Let (Ui,U 2 ) be zero mean jointly Gaussian random variables with variances a\ and a\ respectively 
and correlation p. The channel outputs Y { = Xi + Ni, i — 1, 2 where iVj is Gaussian with zero mean and 
variance a 2 N ^. Also N x and N 2 are independent of each other and also of (Ui,U 2 ). 

In this scenario, the right side of the inequalities in (|23l-(|23T) are maximized by taking Xi ~ 
jV(0, Pi), i = l, 2, independent of each other where Pi is the average transmit power constraint on 
user i. Then I(X h Yi) = 0.5log{l + Pi/a 2 N .), i = 1, 2. 

We can easily specialize the above results to a TDMA, FDMA or CDMA based transmission scheme. 



Thus, now as commented in the last section, for the orthogonal channels (without side information) 
an optimal scheme would be to use Slepian-Wolf source coding and then channel coding each source 
optimally as in a point to point communication. In fact for the Gaussian sources with orthogonal Gaussian 
channels and no side information, vector quantization of sources followed by Slepian-Wolf coding is an 
optimal source coding scheme ([24]) for two user case. This followed by iid Gaussian coded sequences 
with mean and variance Pi, i = 1,2 will provide the overall optimal scheme for this system. The 
optimality is due to the fact that separation holds for this system ([26]). This is the SB scheme we have 
been studying in this paper. 

In the following we compare the performance of the Amplify and Forward (AF) scheme, which makes 
the sensor node design simple, with the SB scheme. Unlike in the GMAC there is no interference between 
the two users when orthogonal channels are used. Therefore, in this case we expect AF to perform quite 
well even at high SNR. 

VII. Amplify and forward over orthogonal channels 

In this section we study the performance of AF in transmitting correlated Gaussian sources over 
orthogonal channels. For minimum mean square distortion, the decoder calculates conditional expectations 
E[Ui\Yi, Y 2 ] and E[U 2 \Yi, Y 2 ). The distortions incurred are given by the respective conditional variances. 
The minimum distortions (Di,D 2 ) are 

n (aia Nl ) 2 [P 2 (l-p 2 ) + a 2 N2 ] (a 2 a N2 ) 2 [Pi(l-p 2 ) + a 2 Nl ] 

1 P 1 P 2 (1 - (?) + a 2 N P x + a 2 N P 2 + « ' 2 P 1 P 2 (1 - ft + a%P, + a 2 N P 2 + a 2 Ni a% 2 ' 

(26) 

From (|26|) we see that as Pi, Pi — > oo the distortions Di,D 2 tend to zero. We also see that D\ and 
D 2 are minimum when the average powers used are Pi and P 2 . These conclusions are in contrast to the 
case of a GMAC where the distortion for the AF does not approach zero as Pi, P 2 — > oo and the optimal 
powers needed may not be Pi and P 2 . 

A. Comparison of AF with SB 

For comparing the performance of the two schemes we consider the symmetric case where P 1 = P 2 = 
P, °\ = o% = v 2 , Di = D 2 = D, a 2 Ni = a 2 N2 = a%. 

For the present system, the minimum distortions achieved in SB and AF are denoted by D(SB) and 
D(AF) respectively, a 2 is taken to be unity without loss of generality. We denote P/a 2 N by S. Then 



D(SB) and D(AF) are 



«^/^ + ^ (27) 

n(AF) S(l-p*) + l 

D{AF) ~ l + 25 + 5 2 (l-p 2 )- (28) 

We see from the above equations that when p = 0, D(SB) = D(AF) = 1/(1 + 5'). At high 5, D(AF) rs 



1/5 and ,0(55) « ^1 - p 2 /5. Eventually both and D(AP) tend to zero as 5 -> oo. When 

5^0 both D(55) and £>(AF) go to a 2 . 

By squaring the equations (O and (EH) we can show that Z?(AF) > D{SB) for all 5. But in the 
following we show that D(AF) is often close to D(SB). 

We note that both D(AF) and D(SB) are lower bounded by 5(1 - p 2 )/(l + 5) 2 . We denote this by 
D(LB). D(AF) is also upper bounded by (1 + 5)/[l + 5(1 - p 2 )] 2 , which we denote by D{UB). Then, 



P 2 

D{AF) - D{SB) < D{UB) - D(LB) < % 



+ T, ^ + 1 



(29) 



5 L(i-p 2 ) 2 (i-P 2 ) 

Thus, we see that for large 5 the difference is small and tends to as 5 — > oo. 
Again, from ([28]), D(AF) < 5(1 - p 2 ) + 1. This gives, 

D{AF) - D(SB) < S « + S ^-^ <S + T ^. (30, 

The right side is small when 5 is small. The difference is 0(5) as 5 — > 0. — D(SB) is less than 

the minimum of the right side in (1291 and (|3Q|) . Also, 

5 2 p 2 [l + 5(l-p 2 )] 



D(AF) - D(5S) < D(AF) - D(LB) < 



(l + 5) 2 (l + 25 + 5 2 (l-p 2 )) 
p 2 [l + 5(l-p 2 )] ^ p 



" (l + 25 + 5 2 (l-p 2 )) " l + 5(l-p 2 ) ~ P ' (31) 

From (E9T) . (1301) and (O we conclude that D(AF) - D(SB) is small when 5 is small or large or 
whenever p is small. 

D(AF) and D(SB) are plotted for p=0.3 and 0.7 using exact computations in Figs [7] and [8j These 
figures confirm the theoretical findings. 
B. Side information 

Let us consider the case when side information Z± is available at encoder i, i = 1, 2 and Z is available 
at the decoder. One use of the side information Zj at the encoders is to increase the correlation between 



the sources. Then while using SB, the encoding rates at the two sources can be decreased. This can be 
optimally done (see [5]), if we take appropriate linear combination of (£/», Z t ) at encoder i and use SB, 
as done in Section |V] 

We provide the comparison of AF with SB for Ui, U 2 ~ A/"(0, 1). Also we take the side information 
which has the same structure as in Section IV-DI 

We have compared AF and SB with different p and s 1 ,s 2 by explicitly computing the minimum 
(D\ + D 2 )/2 achievable. We take Pi = P 2 . Due to lack of space we provide only one case in Fig.9 where 
Si = s 2 = 0.5 and p = 0.4. From the figure one sees that without side information, the performance 
of AF and SB is very close for different SNRs. The difference in their performance increases with side 
information for moderate values of SNR because the effect of the side information is to effectively increase 
the correlation between the sources. Even for these cases at low and high SNRs the performance of AF 
is close to that of SB. 

For the symmetric case discussed here, for SB, encoder-only side information reduces the distortion 
marginally. This happens because a distortion is incurred for (Ui, U 2 ) while making the linear combinations 
(Li, L 2 ). For the AF we actually see no improvement and the optimal linear combination has h\ = b 2 = 0. 
For decoder-only side information the performance is improved for both AF and SB as the side information 
can be used to obtain better estimates of (U±, U 2 ). Adding encoder side information further improves the 
performance only marginally for SB; the AF performance is not improved. 

VIII. Conclusions and Future directions 

We have specialized a general result discussed in [20] to the Gaussian sources and the Gaussian MAC. 
Next, we analyze three joint source-channel coding schemes available in the literature and compare their 
distortion performance. We prove that the amplify and forward scheme (AF) is sub-optimal at high SNR's. 
Also we show that another scheme in [14] is asymptotically optimal as the SNR increases. Cases with side 
information are also addressed. We also provide optimum power allocation policies for the three schemes. 
Next we study orthogonal Gaussian channels. We identify an optimal joint source-channel coding scheme. 
Then we show that unlike the GMAC case, in orthogonal case the AF can often be close to the optimal 
scheme even at high SNR. 

In future, it will be important to study the optimal schemes for moderate values of SNR. Also, of 
course one should obtain optimal schemes for transmission of non Gaussian sources over a GMAC. A 
good scheme has been provided in [19]. 
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SNR vs Distortion performance 




Ql , 1 , , 1 , 1 

-20 -15 -10 -5 5 10 15 20 

SNR (dB) 



Fig. 1. Comparison of performance of the three schemes: SNR vs distortion for p=0. 1. 
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Fig. 2. Comparison of performance of the three schemes: SNR vs distortion for p=0.75. 
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Fig. 3. Distortion for AF for different number of users, p = 0.8. 




TABLE I 

Optimal power allocation for AF for the asymmetric case 
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SNR vs Distortion for AF rho=.1 
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Fig. 5. SNR vs distortion performance for AF with side information, si = S2 = 1, p = .1. 
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Fig. 6. Comparison of three schemes with decoder only side information: Distortion vs side information power, SNR=0dB, p = .5. 
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Fig. 7. Orthogonal channels: SNR vs distortion performance for SB vs AF p = .3. 
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Encoder and decoder side information s1 =s2=0.5 rho=0.4 
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Fig. 9. Orthogonal channels: AF and SB with both encoder and decoder side information, p — .4. 



